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Laser Radar Based Collision Avoidance System for Stationary or Moving Vehicles, 

Automobiles, Boats and Aircraft 
Field of the Invention 

This invention relates to an apparatus capable of developing three-dimensional 
range representations of the objects surrounding a stationary or moving vehicle, with or 
without the presence of obscuration, calculating the likelihood of collisions with these 
objects and warning the operator of these potential collisions. In particular, this 
invention relates to a laser radar device for transmitting laser pulses, collecting the 
O reflected data from surrounding objects, producing the three-dimensional time- 
m dependent object representations and using these dynamic three-dimensional 
y representations in avoiding collisions or minimizing damage resulting from collisions, 
g Background of the Invention 

jr Modern laser radar (ladar), by accurately detecting the time-of-return of 

lI reflected signals from surrounding objects, can not only rapidly construct an accurate 3- 
ry D image of these objects but the range to these objects as well. Furthermore, with the 
proper laser wavelengths, the three-dimensional environment surrounding the ladar 
can be developed even though obscurants such as fog. It is also possible that other 
electromagnetic signals, such as microwaves, can be processed to yield accurate 
three-dimensional information. In addition Doppler-shifted frequencies from continues 
laser reflections can give velocity information about moving objects. 

The rapid development and computer storage of the 3-D physical environment 
surrounding a moving vehicle can be used by an on-board processing computer to 
estimate time-of-impart with portions of other vehicles and warn of potential collisions, 
present the problem situation visually or verbally, suggest collision avoidance or 
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minimal-damage maneuvers, or actively avoid collisions with these vehicles. This 
application of ladar sensors is the complement of militarily using them in the nose of a 
missile to select a target among a number of objects and cause a collision with a 
chosen spot on that target. 

The computer-stored environment on the ladar-equipped vehicle can be updated 
very rapidly so that it always represents the current vehicle's physical environment, 
even during a high-speed collision avoidance maneuver. Where weather-caused (fog 
or snow for example) or man-made obscuration (smoke for example) is present, the 
□ ladar-based collision avoidance system (LBCOS) would be capable of enabling safe 
2 transit though the obscuration. U.S. Patent No. 6,113,989, 3-D Imaging Laser Radar, 
J issued October 17, 2000, to the present inventors, U.S. Patent 5,446,529, 3-D Imaging 
7 Underwater Laser Radar issued August 29, 1995, to the present inventors and U.S. 
P patent application Serial No. 09/449,091, 3-D Ladar Imaging Multiple Target Laser 
H= Radar, filed November 24, 1999, by the present inventors discloses 3-D ladar systems 
nj and sensors which can be used in the LBCOS. 

The advantage of ladar as opposed to radar, for example, in a collision 
avoidance system is the shorter wavelength of the radiation which translates to the 
availability of compact, high-energy, short-pulse, inexpensive laser sources and the 
availability of compact sensors. The shorter wavelength also allows higher two- 
dimensional resolution and more compact optics. Short pulses and more sophisticated 
sensors allow high-range or third-dimensional resolution. In addition, both humans and 
computer-based object recognition algorithms rapidly recognize 3-D images. Modern 
computers are now fast and inexpensive enough to process image information in real 
time. 
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In rapidly enfolding accident situations LBCOS provides distance and velocity 
information to the vehicle operator and/or onboard computer that is impossible for the 
operator to develop with his naked senses. This information and the computational 
processing and analysis of this information allows the operator to make damage- 
avoidance choices he would not otherwise know were available. 

For example, consider a motor vehicle situation where a car in the lane to the 
right of the operator's vehicle attempts to move into the operator's lane and is unaware 
of the operators vehicle. LBCOS might generate a yellow flashing light with the verbal 
warning "look right, sound horn" or if a collision were imminent a red flashing light and, 
Jf if the maneuver were safe, the verbal direction "pull left now'. 

o 

0 Consider another motor vehicle example: an operator in an automobile equipped 
gl with Automatic Breaking Systems (ABS) may stop as fast as possible in the direction of 
J original motion in an accident situation because it is the only option he has time to 

1 consider. However, this may put him in danger of being hit in the rear by a heavy 
J vehicle that he has not seen in his rear view mirror. LBCOS could alert the operator to 

2 the danger and provide the operator with a breaking pattern that avoids both collisions 
5fi or provide the operator with a combined breaking and turning maneuver that avoids 

both collisions. Alternatively by taking over the controls, LBCOS implements a collision- 
avoidance or minimum-harm maneuver. When many vehicles are equipped with 
LBCOS, interaction between the LBCOS systems could even be more effective in 
averting a collision altogether or minimizing the severity of a collision. 

Brief Description of the Present Invention 

In the present invention electromagnetic signals reflected from objects 
surrounding a stationary or moving vehicle, are processed to develop a real-time, 
accurate, three-dimensional computational representation of the physical environment 
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surrounding the moving vehicle. An on-board computer calculates the time of impact 
and warns the operator of the potential collision. Different embodiments of the invention 
present to the operator impact avoidance or damage minimization maneuvers. Still 
other embodiments of the present invention take over the controls and perform the 
selected maneuvers. Another embodiment of the present invention is able to penetrate 
obscurations such as fog and provide the same collision avoidance support to the 
operator with or without the obscuration present. 

A preferred embodiment of the invention uses an eye-safe pulsed laser to 
generate electromagnetic signals and uses laser sensor detection and processing 
electronics to create the data that is used by the computer to develop the computation 
representation of the three-dimensional physical environment of the stationary or 
moving vehicle. 

It is the object of the present invention to provide a device for collision avoidance 
that transmits an electromagnetic signal and creates a three-dimensional computational 
representation of the objects surrounding a stationary or moving vehicle by processing 
the electromagnetic signal reflected from the objects. The three-dimensional 
representation is used to predict collisions and the operator is warned of the potential 
danger. Collision avoidance maneuvers may be presented or automatically 
implemented. 

The device comprises signal transmitter means for transmitting an 
electromagnetic signal to surrounding objects; signal receiver means for collecting 
reflected electromagnetic signals from the objects and developing range and two- 
dimension object data; output electronics means for digitizing and/or transferring the 
object data; data processor means for receiving the digitized data, storing the data, 
computing parameters indicative of potential collisions, deciding the likelihood of 
potential collisions and sending control signals to the drive electronics and 
communications system; drive electronics means for controlling the timing and biasing 
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of the signal transmitter, the signal receiver and the output electronics and operator 
communication means for alerting the operator to potential collisions. 

Brief Description of the Drawings 

Figure 1 is a block diagram of the basic system, a preferred embodiment of the 
Laser Radar Based Collision Avoidance System for Moving Vehicles, Automobiles, 
Boats and Aircraft. 

Figure 2 is a block diagram of the full system, a preferred embodiment of the 
p Laser Radar Based Collision Avoidance System for Moving Vehicles, Automobiles, 
m Boats and Aircraft. 

J*j Figure 3 is a block diagram of a preferred embodiment of the LBCOS sensor. 

p z Figure 4 illustrates the use of LBCOS in a motor vehicle accident situation. 

If Detailed Description of A Preferred Embodiment of The Invention 

Two preferred embodiments of the present invention, the Ladar Based Collision 
Avoidance System (LBCOS) are depicted in Figures 1 and 2. LBCOS is designed to 
generate accurate 3-D representations of the objects, and their range, in all or a faction 
of the space surrounding a stationary or moving vehicle. LBCOS then processes these 
representations to determine collision likelihood with portions of the vehicle and the 
objects. The operator is alerted to a possible collision visually and/or verbally. The 
communication may include a suggested emergency maneuver. The visual 
communication with the operator may include a display of the surrounding objects on a 
screen with or without dynamic and range parameters such as velocity, acceleration 
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and distance. The display may suggest a collision avoidance or minimum damage 
maneuver and monitor the maneuver in real time with continuous feedback to the 
operator. It is also possible that under predetermined circumstances LBCOS could 
take control of the vehicle and implement a collision avoidance or minimum damage 
maneuver. 

Six subsystems make up the preferred embodiment of the basic invention as 
shown in Figure 1 ; one or a plurality of signal transmitters 1_ with one or a plurality of 
pulsed lasers la, transmit optics lb, and a laser pulse detector 1c, one or a plurality of 
M- signal receivers 2 with receiver optics 2a and one or a plurality of sensors 2b, the 

•SK3? 

y output electronics 3, and associated drive electronics 4, the processor 5 with 

JJj processing computer 5a, memory 5b, data base 5c, and software 5d, operator 

g communications system 7 with all or some of visual display 7a, alarm 7b and verbal 

□ directions 7c. The signal transmitters 1_ may be stationary or rotating. Rotation would 
cover a larger solid angle with a lower power laser but would not be appropriate for 

O applications, which require the minimum response time. 

m 

The Ten subsystems make up the preferred embodiment of the invention as shown in 
Figure 2; all the Figure 1 parts as well as the operator interface 8, the environmental 
sensor subsystem 9, the automatic vehicle controls 6, and passenger sensors, 10. The 
Figure 1 embodiment of the invention is designed to be less costly and different 
embodiments will have greater sophistication. For example, the communication system 
7 may lack a visual display 7a component; the software 5c may be less sophisticated 
and processing computer 5a less powerful. In other embodiments of the invention 
signal transmitter and receiver will cover varying percentages of the total solid angle 
relevant to the vehicle application. For example, the physical environment above an 



6 



automobile is not necessary but it may be for an aircraft. The environment underneath 
the vehicle may be relevant to a boat or aircraft but not to a motor vehicle. Additionally 
an individual may feel that only the physical environment to the rear of his automobile 
is relevant and therefore avoid the cost of full 360 degree coverage. The Figure 2 
embodiment is the more advanced system but different embodiments could have 
various levels of sophistication. 

The pulsed laser 1 can be a laser diode with an energy per pulse measured in 
micro-Joules or for longer range systems the pulsed laser can be a flash lamp or diode 
pumped solid state laser with a pulse energy measured in Joules. A preferred 
embodiment is an eye-safe laser wavelength of about 1.57 urn which can be produced 
using an optical parametric oscillator (OPO) with a NdYAG solid state laser. The 
preferred laser pulse detector la is a diode responsive to the laser wavelength 
supported by high bandwidth electronics. The receive optics 2a would typically have an 
aperture between .5 cm to 20 cm, depending upon the application. The transmit optics 
1a may be combined with the receive optics 2a or be separate from the receive optics. 

Typically the output electronics 3 would contain analog to digital converters gain 
and offset correction circuitry, data storage capability and may contain hardwired data 
processing algorithms necessary for high-speed data processing. Typically the drive 
electronics would contain all the sensor 2b biasing circuitry and the master clock 
necessary for operation of the sensor 2b. Typically the signal processor would be 
comprised of a computer mother board containing the processing computer 5a, 
typically an available integrated circuit chip such as an Intel Pentium, associated high- 
speed RAM memory (5b) and data buses, and a high-speed hard drive data base (5c). 
Typically the software 5d would be object-recognition, velocity-computation, time-of- 
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collision-computation, decision-making, user-interface and system control software 
written specifically for the LBCOS application. A preferred visual display 7a is flat panel 
display that can also be used for the operator interface 8. Depending upon the speed 
requirements of the LBCOS application, portions of the software operations may be 
hardwired in the output electronics 3. The environmental sensor subsystem 9 would 
typically contain sensors to evaluate precipitation, speed and road coefficient of 
friction, where applicable. In some circumstances the environmental sensor subsystem 
9 would be in communication with the sensors already in the vehicle which measure 
similar properties. The automatic controls 6 would typically contain all electronic 
boards necessary to communicate with the processing computer 5a and all the motors 
necessary to control the vehicle mechanical systems such as steering and brakes. 
Where automatic pilot systems already exist, such as in sophisticated aircraft, the 
automatic controls would link the LBCOS signal processor 5 directly to the automatic 
pilot computer. In some circumstances the automatic pilot computer may also be the 
LBCOS computer 5a. Passenger sensors 10 typically measure passenger position and 
may measure passenger weight and tightness of restraints such as seat belts. These 
sensors may also be equipped with motors and actuators to modifiy and/or activate 
passenger restraints such as seat belts and air bags. 

Three subsystems make up the preferred embodiment of the LBCOS sensor 2b 
as depicted in Figure 3, the photon detector 13, the electron amplifier 12, and the 
readout integrated circuit (ROIC) JM. There may also be a plurality of ROICs JM 
associated with each sensor. Electrical signals are transferred between the detector 13 
and electron amplifier 12 and between the electron amplifier 12 and ROIC U. These 
three subsystems may be enclosed in a vacuum tube where the detector 13 is a 
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photocathode and the electrical signals that flow between the photon detector 13 and 
the amplifier 12 are vacuum electron current. In the vacuum tube sensor configuration 
the electron amplifier 12 can be a microchannel plate or a solid state detector or a 
solid state detector array used in an impact ionization mode (electrons accelerated 
from the photocathode to the solid state detector array). The photon detector 13 may 
also be combined with the amplifier 12 in an avalanche photodiode array configuration. 

The ROIC 11 is typically an array of unit cells, each unit cell typically containing 
digital and/or analog circuitry for processing and storing data indicative of the range of 
objects in the vehicle's environment which, reflect laser light. The data also typically 
includes the peak amplitude, amplitude time history or a sequential sampling or 
integration of the reflected laser pulse. Typical ROIC unit cell circuitry would be high- 
speed counters, high-speed shift registers, storage capacitors, Schmitt triggers and 
amplifiers. Typical ROIC unit cell array sizes are 1 x 1 to 10,000 x 10,000. The ROIC is 
electrically connected to both the drive electronics 4 and the output electronics 3. 

In another preferred embodiment of the LBCOS sensor 2b, the signal amplifier 
12 would not be present and electrical signals are transferred directly between the 
detector 13 and ROIC U in the form of an electrical current. This transfer could be by 
means of metal bumps directly in contact with the detector 13 and ROIC 11 In the latter 
configuration, typically the detector 13 is a solid state detector array. In some 
circumstances the detector array may have an array of microlenses etched into the 
surface. Typical array sizes are 1 x 1 to 10,000 x 10,000. 

The LBCOS functions as follows: LBCOS's pulsed laser la is continually 
emitting laser pulses at a rate of about 10 to 10,000 Hz. The transmitter optics lb 
directs these laser pulses in all appropriate directions in the vehicles environment. 
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Each laser pulse is designed to illuminate all or a significant percent of the total solid 
angle of concern. Each time a laser la fires a pulse the associated laser detector 1c 
senses the emission and causes a master clock in the drive electronics 4 to begin 
operations in the associated ROIC H of the sensor 2b. Typically when operations 
begin in the associated ROIC JM, a high-speed counter would begin counting in each 
unit cell of the ROIC or a ramp voltage would be input to each ROIC unit cell. For 
underwater imaging, or when obscurants are present in the atmosphere, the ROIC unit 
cell input current would be sequentially integrated and stored or the input current would 
be converted to a voltage with a transimpedance amplifier and the voltage would be 
sequentially sampled and stored. Laser light reflected off surrounding vehicles (vehicle 
2, vehicle 3 and the cement barrier, for example, in Figure 4) would enter the receive 
optics 2a and be focused on the sensor 2b. This light would enter the detector 13 and 
be converted to an electrical current. If an electron amplifier 12 is present this current 
would be amplified. In general this form of amplification, amplification by secondary 
electron emission in a microchannel plate or impact ionization in a solid state material 
is superior to amplification in ROIC circuitry because there is very little noise 
associated with it. An electron amplifier 12 can reduce the power requirements of the 
pulsed laser la. 

Electrical current from either the detector 13 directly or the amplifier 12 enters 
the unit cells of the ROIC It For each laser pulse, each unit cell is associated with a 
specific portion of the solid angle in the surrounding region of concern. Typically all or 
a significant portion of the unit cells on the ROIC 11 are associated with the solid angle 
illuminated by a single laser pulse. If the sensor scans the surrounding region then the 
unit-cell solid angle may change from laser pulse to laser pulse. Typically as the 
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current pulse rises in the RO!C unit cell, a threshold is reached and the ramp voltage or 
range counter is stopped. The ramp voltage at threshold is stored as an analog signal. 
The range counter stores digital data. In addition the pulse amplitude is sampled at 
one or more points and stored as analog data. When a preprogrammed maximum time, 
determined by the expected range in LBCOS application, is reached, readout of the 
ROIC 11 data is begun by the drive electronics 4. This maxiimum time could be 
measured in nanoseconds to microseconds. The ROIC data is transferred to the output 
electronics 3 where it is corrected for gain and offset and typically where the range 
calculation is made. The range and unit-cell position is transferred to the processing 
computer 5 where the software 5d compares the data with previous frames and where a 
velocity and time of impact computation is made by the associated software. Typically 
objects are identified from the data base 5c as they enter the field of view of the 
LBCOS system. Identification is not generally required for each laser pulse. Typically a 
frame is the data gathered with one laser pulse or a minimum number of sequential 
laser pulses, which completely encompasses the environment of concern. 

Various levels of sophistication are possible with the LBCOS. A sophisticated 
signal processor 5 might include object recognition algorithms to determine the nature 
of the potentially colliding vehicle. The signal processor data base 5c could include 
weight of the vehicles, stopping and maneuverability characteristics and parameters. 

The operation of LBCOS is further illustrated by the Figure 4 hypothetical 
accident situation. The driver in vehicle-1, a passenger car, is traveling in lane-1 with 
his small children in the rear seats. There are two other vehicles in his lane, lane-1 , a 
large truck behind and another passenger car in front of vehicle-1 . It is raining and 
vehicle-2, looses control, strikes the concrete divider and begins to spin into lane-2 
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while continuing to move in the direction of traffic. Without LBCOS the driver of vehicle- 
2 would most likely instinctively apply his brakes and stop as soon as possible, 
oblivious to the limited stopping ability of the truck behind him. If vehicle-1 were 
equipped with an automatic breaking system (ABS) and an assisted ABS, collision with 
vehicle-3 would almost be assured since vehicle-1 would stop in the minimum time. 

Prior to the loss of control of vehicle-2, LBCOS may have identified the vehicles. 
It can use these vehicle characteristics to continually monitor the collision danger and 
may have already alerted the vehicle-1 operator of vehicle-2's and vehicle-3's range 
and position by means of the operator communications system 7, informing him of the 
danger of an minimum-distance emergency stop. As vehicle-2 goes out of control, 
LBCOS in vehicle-1 may first alert the operator of danger by means of the alarm 7b in 
his operator communications system 7. This is important for operators who may be 
distracted by passengers or cell phones, for example, and may not be immediately 
aware of the danger. LBCOS determines the danger by calculating and analyzing 
vehicle-2's motion and/or calculating time-to-collision at the current speeds. LBCOS 
may then present to the vehicle 1 operator the breaking pattern required to avoid a 
collision as vehicle-2 slows down in the forward direction. Also being aware of the 
motion of vehicle-3 and its stopping range, LBCOS would recommend by means of the 
visual display 7a or verbal direction 7c that the breaking pattern include collision 
avoidance with vehicle-3. The breaking pattern may be combined with a lateral 
movement into lane 2 or may include an acceleration pattern to pass vehicle 2 once it 
enters lane 2. Verbal directions 7c can be complemented by a visual display 7a that 
presents the danger situation, the suggested maneuver and real-time trajectories 
showing the effectiveness of the maneuver, much like a multi-dimensional video game 
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display. The visual display can be projected on the windshield so the operator's eyes 
are not diverted from the accident situation. 

If collisions were unavoidable LBCOS would recommend a minimum-harm 
maneuver. The collision-avoidance or minimum-harm maneuver would be based upon 
the road conditions as monitored by the environmental sensor subsystem 10, and the 
maneuverability and impact vulnerability of vehicle-1 as stored in the data base 5c. 
Vehicle-1 movements would be continually monitored by LBCOS with continuous 
feedback by means of the communications system 7. Alternatively, if vehicle 1 were 
equipped with automatic controls 6, it would implement the high-speed collision 
avoidance pattern, assuming the vehicle 1 operator had so indicated though the 
operator interface 8 prior to the trip, and return control to the operator when the vehicle 
was out of danger. LBCOS's automatic controls 6 may also be programmed to take 
control when the driver's reaction time to verbal commands is not sufficient to avoid the 
collision. If the vehicle were equipped with passenger sensors 9, LBCOS would take 
passenger position and possibly weight into account when calculating a minimum 
damage maneuver. Otherwise passenger position would have to be input to the 
processor 5 by means of the operator interface 8. LBCOS may also cause passenger 
sensors to tighten seat belts when warranted and deploy air bags at precisely the 
correct time. 



Claims 



What is claimed is: 
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